A number of systems have been developed for providing network communications among groups of users. These systems may employ an architecture, such as a Totem ring network, in order to provide a fault tolerant structure for the network communications. Totem ring networks are relatively well known and provide for multicast delivery of messages, wherein messages may be transmitted and delivered to multiple locations, and ensure that the sequence in which messages are generated is maintained as such messages are transmitted and delivered throughout the system.
The Totem ring protocol operates by organizing the nodes of a system into one or more virtual rings of processors around which a token rotates. When a token is received by a processor, that processor may multicast any messages in its pending transmit queue. The token contains a sequence number and retransmit list.
The sequence number is used to provide a total order of messages such that each receiver can order messages in the same order. On each multicast the token's sequence number is increased. When the token is forwarded to the next node in the ring, the token sequence number is sent based upon what was received plus the number of multi-casted messages.
The retransmit list is used to request retransmission of missing messages. On receipt of a token, the processor compares the token's sequence number with its currently received messages. If any message is missing from its list, it augments the retransmit list with the missing messages. Also, upon receipt of the token, any messages in the retransmit list for which the processor has a copy are multicast by that processor.
In order to support multiple rings, the totem protocol creates gateways between the totem single rings. In general, each of the rings operates on separate multicast target addresses. Each gateway only forwards messages which are required by the other ring. In this way, the totem multi-ring protocol is more scalable. For example, it is possible for each sub-ring among a plurality to obtain the maximum throughput available and reduce latency by ½ the token rotation time in a single ring structure.
In order to determine whether to forward a particular message, the gateway nodes check character string in each message called a “group” that identifies which messages should be sent to which receivers. Each gateway maintains a list of groups that are relevant to the rings it interfaces.
Unfortunately, there are several drawbacks of the known implementations of Totem ring networks. One drawback is that the individual rings in a multi-ring system must be configured ahead of time. Thus, in order to optimize the configuration of the sub-rings, the user setting them up must be aware of the locality of communication occurs or will occur in the multi-ring network. Understandably, this information may not be available or may change frequently during operations of the multi-ring network. Utilizing an inferior setup of the sub-rings will degrade performance of the system due to excessive forwarding of messages across sub-rings.
Another drawback with multi-ring Totem ring networks is that some Totem may be geographically separated over long distances. For example, with the use of wide area networks (WAN), at least one Totem ring may span a long distance, such as a mile or several hundreds of miles. Such a span will likely result in time latencies (such as a few tens of nanoseconds, or more) between nodes on that sub-ring. Indeed, these delays may be sufficiently long to result in undesirable error rates on the Totem rings. Accordingly, it can be appreciated that the configuration of sub-rings can be critical in order to account for factors, such as geographic span of a particular ring or sub-ring.
Accordingly, it would be desirable to provide methods and system that assist in automating the configuration of Totem network rings. It would also be desirable to provide methods and systems that allow the configuration Totem network rings to change dynamically in response to operating conditions.